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THE   COMPARISON  OF  PUPILS'  ACHIEVEMENT   WITH 

THEIR  CAPACITY 

Introduction.  For  the  first  few  rears  after  the  besrinnins:  of  the 
standardized-test  movement,  it  was  the  common  practice  to  interpret 
scores  made  on  achievement  or  subject-matter  tests  without  reference 
to  intelligence  or  other  possible  factors  that  should  have  been  con- 
sidered. Occasionally  an  unusually  thoughtful  worker  suggested  that 
achievement  scores  should  be  compared  with  chronological  ages, 
mental  ages  or  other  measures  of  general  intelligence,  or  perhaps 
something  else  ;  but  it  was  not  until  about  ten  years  ago  that  any 
specific  measures  to  be  used  for  comparing  achievement  with  capacity 
were  suggested.  About  1920.  the  educational  quotient,  the  subject 
quotient,  and  the  achievement  or  accomplishment  quotient  were  pro- 
posed, and  almost  immediately  began  to  receive  widespread  use. 
Since  then,  quite  a  number  of  other  measures  of  the  same  sort  have 
been  proposed.  Moreover,  the  great  majority  of  persons  who  have 
made  use  of  any  of  these  measures  have  done  so  non-critically,  paying 
no,  or  practically  no,  attention  to  their  validity  and  reliability.  Only 
a  few  persons  more  critical  than  most  have  suggested,  and  in  some 
cases  given  evidence  to  support  their  suggestions,  that  these  measures 
do  not  deserve  the  high  degree  of  confidence  that  has  been  placed  in 
them. 

Recommendations  as  to  terminology  and  practice.  On  the  basis 
of  a  careful  study  of  practically  all  written  discussions  of  this  topic, 
and  also  of  his  own  experience  with  measurement,  the  writer  wishes 
to  make  the  following  recommendations.  For  comparing  achievement 
in  a  single  subject  with  capacity  or,  in  other  words,  for  subject  age 
(S.A.)  divided  by  chronological  age  (C.A.),  "subject  quotient"  (S.Q.) 
should  be  used.  If  several  subjects  instead  of  one  are  concerned, 
"educational  quotient"  (E.O.),  which  equals  educational  age  (E.A.) 
divided  by  chronological  age,  is  the  most  suitable  term.  "Achieve- 
ment quotient"  or  "accomplishment  quotient"  (A.Q.)  is  best  limited 
to  the  comparison  of  achievement  with  intelligence ;  that  is,  to 
achievement  or  accomplishment  age  (A. A.)  divided  by  mental  age 
(M.A.).  Of  the  other  quotient,  ratio,  and  difference  measures  sug- 
gested, none  appear  to  merit  acceptance  for  ordinary  school  use  except 
perhaps  Symonds'  "index  of  effort."  It  is  intended  for  use  in  high 
school  and  above,  since  satisfactory  age  norms  to  form  the  basis  of 


achievement  quotients  are  not  there  available.  The  simplest  method 
of  computing  it  is  merely  to  rank  the  pupils  in  any  particular  class  or 
group  according  to  their  intelligence  and  also  according  to  their 
achievement  and  find  the  difference  in  ranks. 

The  validity  of  measures  which  compare  achievement  with  ca- 
pacity. It  has  been  shown  by  several  investigators  that,  from  the 
statistical  standpoint,  the  achievement  quotient  and  most  other  com- 
monly-used measures  of  this  sort  lack  validity.  The  chief  reason  for 
this  is  that  the  two  quantities  that  are  used  as  the  numerator  and 
denominator  of  the  fraction  from  which  a  quotient  is  obtained  are  not 
expressed  in  units  of  the  same  size,  and  therefore  are  not  truly  com- 
parable. At  least  two  suggestions  as  to  how  to  remedy  this  defect 
have  been  made,  but  both  involve  enough  computation  that  they  can 
hardly  be  expected  to  receive  general  use. 

The  reliability  of  measures  that  compare  achievement  with  ca- 
pacity. The  data  presented  in  several  studies  of  this  point,  among 
which  is  one  made  by  the  writer,  indicate  that  the  achievement  quo- 
tient and  other  measures  which  compare  achievement  with  capacity, 
are  not  at  all  highly  reliable.  For  quotients  computed  from  some  of 
the  best  available  standardized  achievement  tests,  the  unreliability  is 
so  great  that  if  the  quotients  that  would  result  from  a  second  applica- 
tion of  the  same  test  are  predicted  from  its  first  application,  the  pre- 
dictions will  almost  certainly  be  more  than  half  pure  guesses.  For 
some  of  the  most  widely-used  tests,  they  will  be  from  four-fifths  to 
nine-tenths  pure  guesses.  Expressing  the  same  fact  in  another  way, 
the  average  errors  in  such  quotients  are  likely  to  run  about  ten  per 
cent  of  the  quotients  themselves.  Therefore,  achievement  quotients 
are  in  most  cases  too  unreliable  to  form  a  satisfactory  basis  of  indi- 
vidual diagnosis,  classification,  or  other  procedure.  If  quotients  are 
based  upon  average  results  from  several  good  tests,  they  may  be 
reliable  enough  to  justify  their  use.  Moreover,  the  average  quotients 
for  classes  or  larger  groups  of  pupils  are  usually  reliable  enough  that 
considerable  confidence  can  be  placed  in  them. 

The  foregoing  discussion  is  a  brief  summary  of  "A  Critical  Study 
of  Measures  of  Achievement  Relative  to  Capacity,"  which  is  being  pub- 
lished as  Bulletin  No.  45  of  the  Bureau  of  Educational  Research,  Uni- 
versity of  Illinois.  It  is  not  being  sent  to  the  complete  mailing  list  of 
the  Bureau,  but  a  copy  will  be  sent  free  of  charge  to  anyone  who  re- 
quests it. 
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